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ABSTRACT 



Many efforts to implement a results-based framework for 
public programs have been marred by confusion about terms and basic 
definitions, and difficulty in identifying appropriate results and 
performance measures. This guide is intended to help planners identifying 
program results and indicators and tie them to an established planning, 
budgeting, and management system in funding for public education and other 
children's projects. The guide presents key characteristics of an effective 
results and indicators list, important steps in developing this list, and 
potential problems that a jurisdiction may face in establishing results and 
indicators and collecting data to measure them. Part One of the guide 
discusses the movement toward results-based accountability and presents the 
rationale for connecting results to budgets. Major shortcomings of current 
budget systems are outlined as well as the task of selecting results and 
indicators in the context of an overall strategic shift to a results-based 
system. Part Two defines basic terms and creates a common vocabulary. This 
part describes a results and indicators list not only as a product, but as a 
process that creates a framework for fundamental change in the way that 
jurisdictions allocate resources. This section concludes with key 
characteristics of an effective results and indicators list that are used to 
inform discussion in the rest of the guide. Part Three raises nine key 
implementation questions that jurisdictions need to ask in order to build a 
politically credible, sustainable, and dynamic process. The section draws on 
the experiences of several states, counties, and cities to illustrate 
problems, choices, and solutions. The guide's brief concluding section offers 
a summary of main points. Contains 19 references. (LBT) 
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PREFACE - ^ 

Public financing for education and an array of other children's services has become a topic of 
“ significant interest and political concern. Growing skepticism among a critical mass of American 
voters and taxpayers has fueled doub.ts about the ability of government to solve problems and 
provide basic supports and services that enh^ce the quality of life in their conmumities. Many 
believe government is too big, that it's too expensive, and that it doesn't work very well. 

Despite steadily increasing public expenditures for health, education, welfare, hi^an 
services, and public safety over the past two decades, 'seemingly intractable problems persist. 
Nearly a quarter of the childr^ in the U.S. are poor and live in families and communities that are 
unable to meet their basic needs. Schools have become increasingly expensive, but student 
achievement has not matched the rising costs and dropout rates remain unacceptably high. 
Health care costs continue to go up, yet many Americans can't get the services they need, and 
with each passing ye^ their health care dollars buy less. Criminal justice demands a 
dramatically increasing share of public dollars— for police officers, judges, and jails — but , 
neighborhood streets don't seem any safer. . \ 

Voters have spoken clearly. They want more for their money — more arid better, services, 
yes, but also balanced budgets and cuts in income and property taxes. After more than a decade 
of chronic deficits' they want government at all levels to operate more effectively and effici^tly. 
They.dori't want to dismantle government, but rather they want government to meet vital public 
needs and make a more visible difference in their lives. 

Elected officials and other policy makers have responded to public concern arid 
dissatisfaction by focusing more explicitly on the results of the programs and initiatives that they 
develop and fund. Reformers have sought to redefine the missions of public; programs and 
agencies, to modify how services are delivered, to measure how well government programs and 
ageiicies are performing, and to feed information about performance back into planning, 
budgeting, management, and accountability systems. While the federal government's National 
. . Performance Review and its initiatives to "reinvent government" may be the most prominent 
examples of this fociis on results, there are coimtless other efforts at the state and local levels that 
span the divisions of ideology, political party, and the executive and legislative branches of 
government. 

Focusing on results is particularly important for programs and policies serving children and . 
their families. The future well-being of the nation is obviously tied to children's healthy 
development. Yet policy makers and citizens alike may be inclined to reduce their commitment 
to critical supports and services without strong evidence that these investments yield results that 
society cares about, such as healthy children, children succeeding in school, strong families,, and 
safe homes and neighborhoods. ' . 

Unfortunately, many of the efforts to implement a results framework — for public programs, 

. generally, as well as those targeted to children and their families — ^have^ been marred by 
confusion about terms and basic definitions, insufficient political imderstanding and support, the 
difficulty of identifying appropriate results and . performance measures, and the challenges of 
overhauling existing planning, budgeting, and management systems. Policy makers trying to 
implement results-based systems have enthusiastically set out in many different directions, but 
often without a particular destination or a map to help them get there. 

The Finance Project, established by a consortium of national fbimdations, conducts aii . 
ambitious agenda of policy research and development activities to improve the effectiveness, 
efficiency, and equity* of public financing, for education and other children's services. Among 
these efforts, is assisting with the important work of achieving and measuring important 
. outcomes for children, their families, arid the communities in which they live. To guide its work 
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in this area, -The Finance Project created a Working Group on Results-Based Planriing, Budgeting, 
Management, and Accountability Systems. 

Under the direction of the working group, a Strate^ Map far Results-Based. Budgeting was 
designed as a road map for those desiring to incorporate results in their planning and budgeting 
systems. The Strategy Map defines results, indicators, and performance measures and offers a 
framework for choosing them. It describes the products and competencies required for designing 
^d putting into place a results-oriented budgeting system and discusses lessons from existing 
initiatives to define, measure, and achieve results. It suggests how to build political and 
commumty support, how to reallocate resources and tie them to results, how to integrate results- 
based budgeting into an existing budgeting process, and how to avoid common pitfalls. It serves 
as a framework for a series of papers and tool Idts for creating ..results-based planning and 
budgeting systems that are under development by The Finance Project: a guide to results and 
indicators, a guide to performance measures, a tool kit on children's budgets, and a paper 
presenting a cost-of-failure/cost-of-bad-results prototype and analysis. 

This paper, A Guide to Selecting Results and Indicators, is one of the tools that the Strategy Map 
spawned. It draws on the experiences of several states, cities, and counties to help guide others 
through the tasks of identifying results and indicators and tying them to an established planiung, 
budgeting, and management system. It lays out key characteristics of an effective results and 
indicators list, the important steps in developing this list, and the potential problems that a 
jurisdiction may face in establishing results and indicators and collecting the data to measure 
them. 

The paper was prepared by Atelia I. Melaville, an independent consultant who has 
researched and written extensively on strategies to improve results for children and their families. 
She and I would like to recognize Bonnie Armstrong, Cheryl Bailey, Janet Bittner, Laurie 
Dopkins, Randy Franke, Mark Friedman, Bev Godwin, Charles Hall, Scott Johnson, Jason Juffras, 
Linda Kohl, Marge Leffler, Ted Mable, Jacqueline McCroskey, Susan Roth, Gaye Smith, Karen 
Stanford, Marvin Weidner, Becky Winslow, Lyle Wray, and Duncan Wyse. The information that 
they provided and their helpful and constructive comments are reflected in the paper that 
follows. 



Cheryl D. Hayes 
Executive Director 
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INTRODUCTION 

The rationale for this series is played out thousands of times each day in hospitals and 
birthing centers around the country. The scene is familiar:' parents meeting their newborn . 
child for the first time. Despite the endless combinations of personal circumstances, cultures, 
and religious beliefs that these families represent, the rush of emotions they experience is 
remarkably similar. There is wonder (he's so perfect!); anticipation (who will she be?); and, 
finally, quiet determination and a profound sense of accountability. It is the moment when 
every parent makes a silent proinise to his or her child to make sure that they have what they 
need to grow up healthy and strong, and to develop their special gifts in their own unique 
way. 

A Guide to Selecting Results and Indicators is one in a series of wqrking papers produced 
by The Finance Project to help communities and governments, in partnership with fainilies 
and neighborhoods, make sure that the essential conditions of success are in place for every 
child. It is aimed at the growing number of states, cities,, counties, and communities ready to 
move beyond good intentions and vague promises to the goal of measurably improving 
' results. The series begins with a conceptual overview: A Strategy Map for Results-Based 
Budgeting: Moving From Theory' to Practice' Subsequent working papers offer specific 

guidance to help cornmunities, forge agreement on the results that they consider most 
important and to develop the tools that they need to liiJc decisions about budgets, programs, 
and policies to a politically credible set of community expectations. 

- This document is the second paper in this series. It is intended' in part, to show the 
hard and gritty work required to bridge the gap between a conceptual approach to results- ^ 
based budgeting and its implementation. It draws on the. stories of nearly a dozen states, 
counties, and cities' which have been going the distance on a daily basis. While this small 
sampling reflects much of the best activity under way across the coimtry, many more 
initiatives not discussed here are doing equally important work. The Finance Project 
welcomes comments from readers of this series about other initiatives and strategies, and 
solutions and innovations, that might be shared in future publications. 

Readers will quickly note that this is not a step-by-step guide. As the experience of . 
these jurisdictions attests, there is no one right way to go about the job of selecting results and 
indicators, or any one set of results and indicators, that is best. But there is a great deal that 
communities can gain from work that has already been done. The Finance Project is grateful 
for the willingness of these pioneers to share what they have learned through trial and error, 
as well as their successes. Their . stories put life in the boxes on the strategy map on page 9 
and they show what is required to move from one "functional plateau'' to another. 

Part One of this document gives an overview of the movement toward results-based 
accoimtability and lays out the rationale for connecting results to budgets. It outlines the 
major shortcomings of current budget systems arid discusses the task of selecting results and 
indicators in ihe context of an overall strategic shift to a results-based system. Part Two 
defines basic terms and creates a common vocabulary. It describes a results-and-indicators 
list not only as a product, but as a process that creates a framework for fundamental change 
in the way that jurisdictions allocate resources. The section concludes with the key 
characteristics of an effective results-and-indicators list that are used to inform the discussion 
in the rest of the guide. Part Three raises nine key implementation questions that 
jurisdictions need to ask in order to build a politically credible, sustainable, and dynamic 
process.' It draws on the experiences of several states, counties, and cities to illustrate 
problems, choices, and solutior\s. A brief concluding section offers a "short list" summary of 
. basic points. , 
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PART ONE: 



MOVING TOWARD RESULTS— AN OVERVIEW 

The movement toward results-based accountability reflects decades-long experience by states 
and localities to answer some key questions: What do we want for our children? What are 
the basic conditions of well-being that all children must have to make the most of their 
potential? Whose job is it to create these conditions? How will we know if weVe got them? 
And, finally, how do we pay for them? 

Getting to these questions has not been easy. They have grown out of the frustration 
that communities and governments at all levels have felt as they have watched seemingly 
intractable problems grow more severe. Despite the continuing input of substantial public 
and private resources by dozens of public and private agencies, too many communities have 
seen test scores and high school completion rates decline, child poverty worsen, and children 
harmed by premature parenting, substance abuse, and violence. 

As they have struggled to find but how they could be trying so hard and yet 
accomplishing so little, states, counties, cities, and communities that are interested in reform 
have come to several important realizations. Together, these, ideas have begun to radically 
change the way that we think about what we want for our children and how we design, 
finance, and evaluate services. 

• First, the most intractable problems facing our children are interrelated. Fragmented 
solutions need to be pieced together into comprehensive strategies. 

• Second, states and communities need to focus more attention on what is happening 
to children, families, and communities than on what agencies and programs are 
doing to and for them. 

• Third, government and public agencies need to work in partnership with families, 
community organizations, and the private sector in order to set new directions and 
see real improvement. 

• Finally, we need to decide on the most important results we want for our children, 
measure our success in achieving those results, and then use that know-how to make 
better decisions about what we pay for. 

The heart of results-based accountability lies in this last idea. If results are things that 
, matter to the long-term well-being of society, then how do we connect them to the work of 
actually deciding how we use our resources? 



The Rationale for Connecting Results to Budgets 

Up until recently, most reform efforts designed to improve results for children and familieis 
have focused primarily on service delivery. Much attention has been given to the elements pf 
effective services and supports, and the way in which existing services could be packaged 
more comprehensively in order to better meet the needs of children and families. There is 
growing recognition, however, that these changes cannot be made without simultaneous 
changes in the way that states and localities finance innovations and manage their budgets. 

All budgeting, is about dividing up available resources , to do certain things. Results- 
based budgeting refers to a budget process that directly connects resource allocation to 
specific, measurable results selected by broad-based agreement among government and 
citizens. It is a process in which budgets are used to drive progress and leverage 
accountability, rather than simply to maintain the status quo. 
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Limitations of Current Budget Systems^ 

To a large extent> our current budget systems suffer from ^d, contribute to many of the 
same limitations that plague service delivery systems. Most public budget processes are: 

• Shortsighted: One and two-year funding cycles encourage short-term solutions. The 
effects of spending decisions — ^not to mention the costs pf failing to provide effective 

. solutions to specific problems— are seldom tracked over multiple budget cycles. In^ 
most budget offices; there is neither the time nor the inclination to develop 
information systems that support long-term strategic planning. As a result, budget 
systems do little to encourage comprehensive, long-term investments in children, 
families, and communities. ^ 

• Fragmented: A myriad of funding committees shepherd their own set of agencies 
' and programs through the budget process, with very little information or interest in 

what one pother is doing. This approach makes it difficult to make comprehensive 
and coherent funding decisions that maximize the impact of expenditures in given . 
" areas. 

• . Focused on inputs rather than results: Current budget systems and their 

mformation systems are designed to track how much agencies and programs spend 
and what they do, rather than whether.they are making tangible improvements in the 
lives of children and farhilies. Budget departments ask if programs are well run, but 
the departments have little way, of knowing if the programs are making any 
difference. 

Why Shifting to Results-based Budgeting Makes Sense 

We make budgets to make sure that we can pay our way. But the question that current 
budget systems fail to ask is: .Pay our way to where? Our inability to answer this question 
has helped to create palpable anger.and resentment among citizens who feel overtaxed and 
uiiderserved. One professional observer of attitudes toward government commented 
recently: "What drives people wild with frustration is the lack of responsiveness, a feeling of 
being i^ored,, misunderstood, exploited, and played upon like a . pack of fools."^ Voters 
don't just want more services for less money, they want some things done much better. And 
they are angry with a government that seems to offer so little help in making reasoned 
choices. ' 

Elected officials are frustrated as well with "the paradox of programs claiming success 
.while conditions get worse "“* for too many children, families, and communities. True, there is 
every incentive for people who. face re-election every two to sbc years to shy away from 
supporting long-term investments that might not bring any itruneidiate political payoff. But, 
to a large extent, elected officials are hamstrung by the lack of strategic tools needed in order 
to mak^ better decisions on behalf of their i constituents. Legislators too often end up micro- 
managing because much ^ of the information that they have at their disposal delivers up 
administrative minutiae rather than’ cogent analysis of broad trends and emerging issues. A 
system of results-based budgeting can begin to supply policy-makers with the tools that they 
need to respond more effectively to what communities want as well as provide them with the 
political support that is needed to make tough choices. 

Using a Strategy Map 

If results are things that matter to the long-term well-being of our society, how do we cormect 
them to the work of actually deciding on how we use our resources? A variety of states and 
ibcalities are exploring this question and learning from each other. Their experience 
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underscores the fact that moving from a system based on problems and measured by inputs 
to one based on results is a complex, multi-year undertakmg. 

One way to approach an undertaking of this magnitude is to devise a strategy map: a 
format that lays out the implementation of a complex effort over time. As the diagram on 
. ^page 9 suggests, states and localities need to move toward results-based accountability, on 
several tracks, refining their competency and the sophistication of their products as they go 
along. First, they must decide what results they want to achieve and how they will measiure 
their progress. Second, they must develop better decision-making tools for tracking progress, 
expenditures, and the costs of bad results, and then decide on strategies that will achieve the 
results they want. And, finally, they must develop a more effective process for using results 
to make key funding decisions. 

This guide focuses on the uppermost track — selecting results and indicators. The final 
adoption of a politically credible list creates a framework within which the work on each of 
the other tracks can proceed. Most directly, selectipn of results and indicators leads to the 
creation of the first of several essential decision-making tools, an indicators report. 

As the strategy map shows, the specificity and utility of this tool will develop over time. 
In many communities, an indicators report will begin as an annual, "single point in time" 
status report. Using a constantly evolving framework of results and indicators, such a report 
can be expanded to include baselines and forecasting data. Eventually, monthly or quarterly 
reports may be developed to show progress toward "turning the curve" on key indicators in 
positive, long-term directions. 

PART TWO: 

THE FUNDAMENTALS OF A RESULTS-AND-INDICATORS LIST: 

DEFINITIONS AND ESSENTIAL CHARACTERISTICS 

Defining Terms 

Developing a. results-and-indicators framework is an ongoing, collaborative effort that, 
requires both technical rigor and political acumen: In any such collaboration, clear 

communication based on shared language is a rock-bottom necessity. 

In the early stages of thinking about moving toward results-based budgeting, language 
issues may seem minor. Shakespeare wrote that ."arose.by any other name would smell as 
sweet." To some extent, the same can be said of results, whether they are called outcomes, 
results, or benchmarks. All these terms are used to mean the same thing: a basic condition of 
welhbeing that people agree they want to achieve. . 

Communication, however, gets much more difficult when people actually start using 
these terms in strategic planning and budgeting. This is especially true as additional terms 
are added to the mix — like indicator, performance measure, outcomes budgets, targets, goals, 
and so on. The best guideline is to agree on working definitions in advance and to use them 
consistently. When this has not been done and confusion builds, do not hesitate to recognize 
and address the problem. 

• In Hampton, Virginia, participants involved in selecting results and indicators were 
all talking about the same ideas, but they were using different vocabularies 
developed in prior work with other groups. Each person was used to thinking and 
talking in his or her own "language." Over time, these verbal preferences had taken 
on a sense of rightness that made thein hard to give up. When members of the group 
finally realized how often they were having to interrupt each other to ask, "What you 
really mean is...?", they decided that "even smart people needed to learn." A 
. subcommittee was established m order to develop a list of common working 
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definitions. The full group talked out their issues, and a common lexicon for results- 
based work was adopted throughoutcity government. . ^ 

• Separate vocabularies also make it more difficult for jurisdictions working on results- 
based budgeting to learn from one another. The jurisdictions described in this guide 
. all use the same basic concepts of results-based budgeting, but use different terms to 
describe them. Georgia refers to 'benchmarks/' Minnesota talte about "milestones," \ 
while Rochester uses "outcomes," So, in order to make sure that, we are all speaking 
- the same language, the next section lays out the key terms and definitions used in the 
. rest of this guide. ' 

Result - , ^ ; 

A "result" is a bottom-line condition of well-being for children, families, or communities. It is , . 
a broadly defined, fundamental condition that government and citizens consider essmtial for 
all its members. One such bottom-line expectation oi the community might be that all of its 
children should be bom healthy. , Another might be, that all children should enter school 
ready to learn. 'A third might be that yoimg people shoulci make a smoo^ transition to 
adulthood. Results are umbrella statements that capture the comprehensive set of needs that 
must be met to achieve success. By definition, achieving these basic conditions of success 
requires concerted action by all sectors of the commuriity. , . 

Some states and communities use the term "outcome" instead of "result/' The meaning 
is the same. However, we prefer to use the term "residt" because it is a less jargon-like and 
'more everyday term, and because it avoids potential confusion with linrelated debates about ; 
outcbmes-based education. i 

Indicator . ' ' 

With respect to developing a results^and-indicators frarhewbrk, we define an indicator as a 
measure, for which we have data, that gauges community-level progress toward agreed- 
upon results! Because results are broad statements of what communities want for their 
children, no single indicator is likely to signal full attainment of any given result. For 
example, Rochester's Change Collaborative agreed that reducing the incidence of low-birth- 
weight babies, improving prenatal care, md reducing the number of births to teen mothers 
would effectively track progress . toward the result of healthy births/ Communities must 
decide what constellation of indicators add up to progress on each result and then require, a 
community- wide, cross-agency effort. . . ' . 

Some initiatives, borrowing from the corporate sector, use the term "benchmark" when 
speaking about "indicator.'' , Here, too, we opt for simplicity and ease of, usage. Because 
"benchmarking" is a term of art in the corporate world and can mean different things to . 
different people, we favor the terai "indicator:"' \ 

What Results and Indicators are 

In order to use these key terms, and the concepts they imply with precision, it is important to 
distinguish them from other commonly used terms ^at have very different meanings. An 
important example is the term "performance measure," It is often confused with the word 
"indicator," even though its meaning vis-a-vis "results;based budgeting" is not the same. 

Performance Measure ; ^ ^ 

Performance measures reflect the achievement of agencies arid specific programs. As such, 
they gauge progress at the agency level rather than at the community level. Appropriate 
performance measures are closely related to an agency's mission and purpose, and are within 
its ability to control. They are narrow measures of how well programs operate with their 
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service populations as part of a larger strategy to ^achieve results for the whole population. 
Examples of perform^ce measures are the number of welfare mothers placed in job training 
programs, or the rate of timely child welfare investigations. 

Failing to distinguish between commuruty-level and agency-level measures of progress 
can. make adopting a results-indicator framework decidedly more difficult. The first 
challenge is to define the best — and relatively few— indicators for each broad result in which 
multiple se^ents of the commuruty. have a part to play. Defining and measuring the exact 
coritributions to be made by individual agencies and programs is an entirely separate process. 
It is essential, however, that the two processes be closely coordinated and that agency-level 
measures reflect actions that will help achieve cominuruty-wide results-and-indicatprs. How 
to craft performance measures within a results-and-indicators framework is discussed in 
another document in the A Guide to Developing and Using Performance Measures in Results-Based 
Budgeting. 

A Framework for Change 

A results-and-indicators list is the linchpin of results-based budgeting: All the other activities 
and decision-making tools that are necessary to build a results-based system flow from its 
-basic premises. Calculations of current expenditures, as well as the costs of continuing 
current trends are made on the basis of agreed-upon results and indicators. Interventions and 
programmatic strategies are determined by research into "best practices" that show clear 
evidence of impact on key indicators. And, ultimately, decisions about where to direct 
dollars are made on agencies' demonstrated ability to produce results. 

A results-and-indicators list is both a product ahd an ongoing process. As a product, it 
stands as a clear, manageable, and politically credible set of the most important results that a 
community wants for its children. It provides “a framework leading to budget tools that 
establish baselines and track progress iri each area on a regular basis. In short, it acts as a 
compass for policy-makers and civic leaders to use in strategic planning, and it provides 
them with basic tools for linking planning to budgets. 

As a process, adopting a results-and-indicators list engages government and community 
in an ongoing , conversation about their expectations for children and families, ^d how 
limited resources should be used. It establishes a relationship based on shared responsibility 
for. what happens to the community's children and families, while at the same time it 
provides a mechanism for fairly distributing institutional accountability. Its continual 
revision reflects changing community priorities and ensures that the most compelling and 
statistically reliable set of indicators is used to measure progress. 



Characteristics of an Effective Results-and-indicators List 

As states and localities work to adopt a results-and-indicators list, they should keep both 
product ^d process in mind. Experience suggests that the most effective frameworks are 
manageable, coherent, persuasive, strength-based, politically credible, and responsive to local 
variation. These six characteristics should inform the final list of results-and-indicators, as 
well as the process that creates if and keeps it alive. 

• Manageable: As one private-sector leader involved in developing a results-and- 
indicators framework put it: "What I want is something I can carry around in my 
pocket and u^." The number of results and indicators should be small enough to 
summarize community expectations in key areas on a single page if possible — and to 
require no rhore than a reasonable outlay of resources to track on a frequent basis. 

• Coherent: Taken together, the results and indicators that form the framework should 
convey a simple but complete picture of community expectations. The selection of 
the results and indicators should suggest comprehensive, cross-cutting strategies. In 
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a well-designed list, the relationship betvyeen each result and each indicator is clear 
and unambiguous, and the conceptual distinctions betw^n results and indicators, 
and performance measures, are clearly defined and consistently used. 

• Persuasive: An effective list should ring true and make sense to people. Results 
should reflect the basic conditions that everyone — regardless of income, race, 
ethnicity, or religion — wants for cMdren and families. Indicators should capture the 
most "common sense" measures of whether we are reaching the desired results. The 

' language used should be as simple and as brief as possible. The response it should 
call forth is: Yes! 

• Strength-based: The tone ^d presentation should emphasize the importance of 

positive youth development and long-term investment strategies, as well as short- 
and long-term remedia: tion. ^ 

• Politically credible: To become a useful budget tool, a results-and-indicators list 
must be recognized as a legitimate statement of what an entire community- — not just 
its government or public agencies— thii^ is essential for children and families. At 
the same time, , the list must be owned and embraced by the executive and legislative 
institutions responsible for setting public policy and financing its activities. 

• Responsive: Whether a list is developed by a state or locality, it must be of value to a 
* , wide variety of users. It should allow for local variation arid should use indicators 

that can be measured with sub-state data whenever possible. 

PART THREE: 

ADOPTING A FRAMEWORK: KEY IMPLEMENTATION QUESTIONS 

Negotiating a Results-and-Indicators Framework, 

In. recent months and years, m^y state, county, and local jurisdictions have developed 
results-and-indicators frameworks. Some are beginning the long-term challenge of 
incorporating them into their systems. A combination of factors have affected both the 
process and the substance of their efforts. .As in any collaborative undertaking, not the least 
of these have been the people involved, the resources available to them, the, needs of, their 
communities, arid the prevailing social, economic and poUtical climate in which they have 
taken place. 

Regardless of where and how they begin, initiatives working to adopt a politically 
credible set of results and indicators face a similar set of questions about how to proceed. In 
the following section, we address several key implementation issues. Many of these 
questions raise overlapping, rather than entirely separate and distinct, issues. Beaimg in 
mind the characteristics of .an effective residts-and-indicators list, we lay out possible 
strategies in each area and draw on the experience of states and localities to illustrate - 
problems, choices, and solutions. • 

Question #1: Who's in Charge? 

The work of developing an effective results-^d-indicators framework needs an "institutional 
champion" — a policy-level body that sets broad direction and expectations for the project,, 
.makes whatever decisions are necessary to ensure that the final product "makes sense," and 
formally adopts the final list. This oversight body must have the political clout and 
legitimacy that are necessary to connect the work done in developing resiilts and indicators to 



ERIC 



14 



THE FINANCE PROJECT 



an overall strategy for shifting to results-based budgeting. The authority and membership of 
this body are critical. 

Ideally, the oversight body in charge of adopting a community-wide set of results and 
indicators is authorized by both the executive and legislative branches of state or local 
government. Its membership draws from both majority and minority parties, and it 
represents a diverse ^oup of civic, corporate, and government leaders, as well as 
representatives of consumer groups. The best "institutional champions" are those that are 
able to stay in business long enough and to accumulate enough political capital to manage a 
complex, long-term change strategy. . Formal standing in state law, buy-in from both sides of 
the aisle, and strong private-sector and community participation are the best guarantees of 
sustainability. In short, an effective oversight body should aim for standing within both the 
executive and legislative branches of government; bipartisan support; and a broadly inclusive 
membership. 

. • A good example of the sustainability possible in this kind of "hybrid" oversight body 
is the Oregon Progress Board. Created by the legislature in 1989 to translate "Oregon 
Shines," the state's long-term strategic plan; into measurable results arid indicators of 
progress, it was conceived by the Governor as "the long-term caretaker of the state's 
strategic vision."^^ It was designed as a nine-member public/private citizen's body, 
chaired by the Governor and subject to biannual reauthorization. The Progress Board 
is currently being chaired by its third Governor and is in its third legislative cycle. 

Not all efforts to develop results and indicators are overseen by institutions so fully 
formed as this. Political credibility and sustainability are possible, but more difficult to 
achieve, in initiatives lodged solely in the executive or the legislative branch. Where 
initiatives do enjoy both executive and legislative authority, this has often evolved over time. 

• In Vermont, work on results arid indicators is firmly lodged in the executive branch 
of state govemihent. It is overseen by a cabinet-level group within the Agency for 
Human Services (AHS) — an eight-department, consolidated bureau responsible for a 
broad array of children's services^and its interagency State Team for Children and 
Families, composed of senior-level staff from each AHS department and the 
Department of Education. Operating without statutory authority, the state team has 
seen its \yprk gain, momentum and political capital, in part from close collaboration 
between the executives of both departments as well as their well-publicized efforts 
throughout the state to promote comprehensive children's services.^ 

• The Georgia Policy Council for Children and Families began life as a citizen's panel 
appointed by executive order. Its 21 members were widely drawn from government, 
business, advocacy, and political circles, but it had no assurance of continuation 
beyond the aorrent Governor's tenure. In 1994, the panel released its five-point plan 
for improving results for the state's children and families, which built on the findings 
of a variety of reform efforts. It simultaneously called for the creation of a state-level 
policy council with the "responsibility and authority to define the results to be 
achieved, to implement the needed strategic policy and systems changes, and to 
monitor process."^ Work went ahead to develop results and indicators while 
legislative efforts were laimched to anchor the Council in statute. In 1995, the 
Georgia Assembly passed the Georgia Policy Couricil for Children and Families Act. 
It created a permanent state body that adopted ^d has continued the work begun by 
the initial group. 

It is also not entirely necessary that framework efforts be located within government. In 
some cases, community coUaboratives, United Ways, and other civic and non-governmental 
organizations are developing results and indicators. However, without institutional 
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participation and governmental sanction, establishing community-wide credibility is much 
more difficult. . 

• In Lamoille County, Vermont, People in Partnership (PIP), is an umbrella group that 
aims to provide a unified voice for families and providers. PIP has been successful in 
encouraging service providers to pool fundirig and engage in joint training. But, says 
^e group's coordinator, "There's a perception that because we're not an official 
nonprofit organization or bureaucracy, that we're not legitimate."'' Even though the 
group has made substantial headway in devising a local framework to shape 
priorities— using Vermont's State Team efforts as a guide — after 10 months, broad- 
based community support is still developing. 

Question #2: Who Does the Work? 

A working group needs to manage the process of selecting results and indicators, identifying 
data measures, and making recommendations to the oversight body for review, comment, 
and eventual approval. On a day-to-day basis the group must introduce discipline into what 
can be. a daimting political and technical task. The work is politically daimting because 
difficult decisions must be made about what results and indicators should be recommended: 
It can be technically challenging, because the data necessary to measiire progress are; 
scattered across dozens of public agencies at various goverrunental levels. An effective 
working group needs technical expertise to help identify, and evaluate complex and often 
conflicting data;, skillful leadership to facilitate the process and keep it moving; and adequate 
staff to support work that is extensive and exacting. 

Working groups vary widely in the number and kind of people they involve. Initiatives 
in Los Angeles Coimty and Rochester, New York illustrate twp widely different approaches: 
i) public/private-sector task forces and 2) interagency staff teams. 

• The Los Angeles County Children's Planning Council chose the former approach. It 
convened a 27-member interdisciplinary team in order to discuss principles and 

^ criteria for selecting results and to make recommendations oh the results and 
indicators the county should track. Because it- realized the complex efforts needed to 
improve results for the children, of Los Angeles, it consciously sought to involve 
people with very different perspectives, training, and experience. The committee 
. was chaired by a university-based member of the Planning Council and included 
^ participation from public agencies, advocacy, groups, higher education, foundations, 
the United Way, ^d the press. Data experts were included in the team, but did hot 
dominate it. A consultant to the Planning Council on community planning processes 
was also available.*” * ^ 

• The Rochester Change Collaborative has relied on the work of ^ interagency staff 
team. The Change Coordinating Team is coihposed of mid-level managers from the 
Collaborative's key partners: the City of Rochester, the County of Monroe, the 
Rochester City School District and the United Way. A Subcommittee on Outcome 

' Measures, representing each of the partner systems, managed the work. The 

subcommittee involved their respective systems' data collection and research and. 
evaluation experts, as well as private-sector analysts. The design and composition of 
Rochester's choice reflect the systems-based nature of Rochester's Change 
Collaborative. It was also motivated by Rochester's conscious decision not to 
"delegate out" responsibility for reengineering its systems to an intermediary group. 
Instead, Rochester officials chose to locate responsibiUty for changing priorities and 
• setting new directions directly within the systems that must do the changing. This 
approach was a way to accomplish what the Change Collaborative was committed to 
doing: reorganizing their priorities and working together. 
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These two approaches each have strengths and weaknesses. Interagency staff groups 
may be more easily convened, and may operate more efficiently, than citizen-based panels. 
Large, public /private-sector working groups may require especially artful leadership in 
order to avoid getting bogged down in irreconcilable differences. As one state official 
working to implement change put it, they also require strong staff "with a temperament for 
testing ideas in public; who can work back and forth with people who are busy and who can 
listen hard and know what to forget." Public/private-sector working groups have the 
important advantage, however, of bringing a broad range of community perspectives more 
directly into the decision-making process. 

Question #3: Where Do We Start? 

How does a working group tackle the job of developing results and indicators? The final 
product needs to reflect community values, but the process must be manageable and finished 
in a reasonable amount of time. Leaders must consider whether the working group needs to 
start from scratch to generate a set of results and indicators, and must determine how 
inclusive the process will be, or whether it can begin with a working list based on the work 
done in other states and localities, and* adapt it as needed. Some of the factors that can make 
a difference have to do with the scope of the initiative and whether prior work has been done 
on developing a basic vision and/or results. 

Starting from Scratch ^ 

Mirmesota and Florida provide two variants on the "starting from scratch" approach. Both 
initiatives had a very broad focus. Each knew that they wanted to identify where the state 
wanted to go across a full range of government activities, not just those having to do with 
children and families. These, initiatives were designed to establish their state's overall vision 
for the future, as well as to select results and indicators by which to measure progress. 
Mirmesota's largely staff-led effort developed a labor-intensive, bottom-up strategy to 
develop a statewide vision that citizens would feel that they owned. In Florida, its citizen-led 
working group took on primary responsibility for establishing the parameters of a state 
vision, with input from members of the public at open working meetings. 

• In early 1991, Minnesota Planning, a strategic and long-range planning agency with a 
director appointed by the Governor, was charged with creating a long-range plan for 
the state based on results,, and stimulating public interest and participation in the 
process. An advisory group including 11 citizens was formed to giye guidance and 
suggestions, but authority for the process and final decisions was located in 
Minnesota Plarmirig. 

• This largely staff-led effort decided to go with^ "a blank piece of paper" and talk 
directly to as many Minnesotans as possible about their vision of the future. In a 
process that lasted more than a year, well-publicized town meetings were held in 15. 
locations across the state. Trained staff facilitators used newspapers and discussions 
to identify major themes important to most citizens. The staff then compiled these 
lists into a draft document that included results and some indicators. After a six- 
month period of community review and revision, 20 broad results (called "goals") 
and 79 indicators (referred to as "milestones") were selected. Some 3,000 people 

' participated in. public meetings, and more than 10,000 played a part in the 
development and review of the first Minnesota Milestones report. 

• Florida's 15-member citizens' Commission on Government Accoimtability to the 
People (GAP) was appointed in 1992 by the Governor an.d established by statute in 
1994. GAP's Benchmark Committee, chaired by a corporate CEO, also chose To start 
from scratch in developing a broad set of themes. In a modification of Mirmesota's 
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approach, however, the areas and topics in Florida were developed by the committee , 
itself — in open meetings that welcomed public participation— rather than culled from 
an extensive set of community forums. In early 1996, the Florida Benchmarks Report 
/ was issued, covering seven major areas of concern, with 134 results (called "topics") 
and 268 quantitative indicators (called "benchmarks"). 

Using a Working List 

For initiatives concerned primarily with child, and family issues— and where there is a broad 
vision and perhaps a set of core results in place — it makes sense to take advantage of the 
< maiiy individual lists of results and indicators that have. already been developed. This guide,^ 
for example, contains several. Many of them have notable similarities and address 
fundamental concerns that are important to most communities. Other suggested lists with 
useful aimotations have been developed by national groups.^* Internet access can expedite 
finding this informatiorl.'^ Constructing a working list based on examples of results and 
indicators developed in other jurisdictions can help to jump-start working group discussions, ' 

. give them focus, ^d save time. 

In Los Angeles County, for example, the working group began with a suggested list of 
results and indicators developed by Joining Forces, a national project to support collaboration 
between education and. social vveliare sectors. In Georgia, a long list of possible indicators 
. based on a. graduate student's literature review of indicators efforts across the country 
became one of the Task Force's basic working documents. A number of jurisdictions used 
national and state-lever Kids Count reports to take adv^tage of important existing work on 
developing indicators. 

Building a working list, however, should not only incorporate indicators from nationcd 
lists like IGds Count or those developed by other jurisdictions. As we discuss more hilly 
under Question #7, it is imperative that working lists reflect local work on developing 
indicators as well. Ir\ numerous communities, groups like the United Way and community- 
based collaboratives may have already made significant progress toward a thoughtful 
indicators list. - . . ' . 

^ It is also important to remember that developing a working list is only the first step m 
building a politically credible framework. As the remainder of this guide makes clear, 
debating and reshaping these Usts, evaluating them according to the group's ovyn criteria, 
setting priorities, and winning broad-based acceptance for the fined product are what will 
make the process real. It makes sense to benefit from work that has. already been done , on 
developing' indicators; however, as one city participant put it: "We soon found out that 
everything we needed couldn't be borrowed." 

Question #4: What Criteria Should We Use in Selecting R<esults and Indicators? 

Working lists can quickly become laundry lists. A wish list of results can grow with dizzying 
speed. The number of indicators can easily outstrip the mimber of results by a factor of 10, 
20, or more. Everyone at the table has their own insight on what is needed to measure 
progress, or a special piece of data they vyant to use. But choices must filially be made. The 
final, list musf not only be short enough to be easily used it should be logically complete. It 
must communicate powerfully and clearly to many audiences, and emphasize the potential of ; 
children and families, as well as problems. \ 

Using some variant of the following set of criteria can help to ensvire an effective final 
product. Every item that stays on the feal list should stack up well against the following 
criteria: 
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Communication Power 

Each result and indicator must "strike a chord" with everyone from parents to politiciai\s. 
Results and indicators, must help to convey a , commitment that is readily uriderstood, 
positive, and pragmatic. Some questions to ask in evaluating potential results and indicators 
include: 

1. Do the indicators pass the "public square" test? Imagine standing in a public square 

‘ , and having to explain to a crowd of people each one of the bottom-line conditions of 

well-being for children that are on your list. Which few pieces of data would you use 
to clearly explain each result — quickly and before the crowd got tired of listening? 

2. Would the results capture what we want for our children, not what we don't waiit? 

3. Are they essential? Results should capture the fundamental conditions of well-being 
that we want for all children, rather than an exhaustive list of every possible 
advantage. 

• in Minnesota, for example, one indicator that emerged from commimity meetings 
called for the "retention of the family farm." Minnesota's Planning's Advisory Team 
questioned its inclusion. Although, this indicator was important to some citizens, it 
was considered less than critical for the state as a whole, and subsequently was 
dropped from the list. 

Proxy (or Predictor) Power 

This criterion looks for results and indicators that wUl fit together into a coherent framework. 
It evaluates whether — and how well— an indicator serves as a "stand-in" for 1) a result and 2) 
other indicators moving in the same direction. It asks: . 

1. Is there a strong and established relationship betvveen the indicator and the result.it is 
intended to measure? Would improvements made on the indicator be accepted as 
reasonable approximations of progress toward the result? In other words, is the 
indicator a good predictor of the result it is intended to measure? Is there research 
evidence to support this connection? If not, is there common-sense linkage? 
Emerging research, for example, suggests that 3”^-grade reading scores show a strong 
correlation with graduation rates. Given the choice, it would make more sense to 
pick this measure as an indicator of school success than, say, the percent, of children 
enrolled in Chapter 1, a much less powerful predictor. 

2. To what extent does the indicator act as a proxy for other indicators moving in the 
same direction? Lengthy indicator lists can be substantially pared down by looking 
for the one major indicator in which movement will mean progress in a herd of lesser 
ones. 

• In Hampton, Virginia, the Family Resource Task Force considered using the rate of 
^ substance abuse by pregnant women as an indicator for healthy children. 

Eventually, however, the group decided that there was not enough research to 
reliably support the correlation. They also decided that using low birth-weight as an 
indicator would exercise a far greater "herd effect." 



Data Power 

This criterion evaluates indicators based on whether data to measure them are 1) valid and 
reliable, that is, that they accurately and consistently measure what they say ^ey measure; 2) 
routinely collected at an appropriate level; and 3) accessible witliout a significant time lag. 
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As the following illustrations suggest, and as we discuss in the next section, finding indictors 
of sufficient quality ^d timeliness is not always easy. 

• Vermont's State Team for Qiildren arid Families was anxious to use domestic 
violence as an indicator of family stability, but could not find. an adequate measwe. 
Court records were suggested. However, bn consideration, it became clear that large 
numbers oif cases were settled, dropped, or dismissed before they ever c^e to court. 
Thus, the measure of court records was viewed both as unreliable and vastly imder- 

. representative. The working ^oup decided to keep looking. 

• Minnesota Planning initially thought about using the number of people u tilizin g food 
closets as a measure of well-being. On second thought they realized that utilization 

, rates could be affected as much by the availability and access of the pantries 
themselves as the need of the people using them. They realized that they needed 
more a valid measiu’e. ^ 

We encourage readers of this guide to come up with their own criteria, keejping in mind 
the importance of communication power, proxy power, and data power. Other discussions 
of criteria useful in selecting results for c^dren are /available and may be helpful to 
jurisdictions in refining their own thinking.^^ The following list notes the, criteria of some of 
the initiatives profiled in this guide: 

• The Georgia Qiildren's Policy Council agreed, to a set of four criteria including: 1) a 
bias toward prevention; 2) strong scientific evidence; 3) available data updated 
regularly at the county level; and 4) compatibility with other state results lists (e.g.. 
Kids Count). , 

• Vermont's Interagency State Team developed a three-criterion list. They looked for 
indicators that 1) are readily imderstood by the public; 2) act as proxies for other 

, indicators; and 3) for which reliable data are available, at the supervisory iinion (or 
school district) level, if possible, preferably on an annual basis. 

• Florida's GAP Commission developed a nine-item set of criteria drawn from the 
performance-management literature. The list included outcomes-orientation, 
reliability, availability, accuracy, utility, comparability, and sensitivity. 

•. The Los Angeles County Children's F’lanning ' Coimcil's principles for outcome 
measurement can easily be . translated into criteria. The hill set of principles is 
. included in the Appendix. . 

Question*#5: What are Common Problems in Data Collection, and How Can We H^dle 
Them? 

We are well aware that the neat and tidy selection criteria just discussed — even with the brief 
"stories" vye have included as illustrations— do not convey how arduous a task it can be to 
sort through potential indicators. In many cases, a good bit of legwork will be needed to 
track down potential measures, which may or may not turn out to be the right ones. There 
are two major difficulties in finding a sufficiently powerful set of results and indicators: The 
first has to do with managing the sheer volume and complexity of the data; the second arises 
froiii the nature of the data themselves. . : ^ 

■ ■ ^ . ■ * ' ■ 

• Rochester's Change Coordinating Team,, for example. Wanted to use the percentage 

of 16-to-24 year-olds in school or working as an indicator of yoimg people avoiding 
risk behaviors. They selected it because it represented a broad arid positive measure, 
and assumed that the school district would have the data. The school district 
' representative did some checking and foimd that the^ school district did not. 
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Inquiries made at the U.S. Department of Labor also turned up empty. The source of 
the indicator was finally located at the U.S. Census Department. After considerable 
searching/committee members realized that the data needed to measure the indicator 
were collected only once every ten years and would take a number of years to be 
reported. They decided that they needed an indicator that they could track much 
more frequently. 

Managing the Data 

Getting a handle on the data that is out there — ^not to mention the data is needed — is no 
small matter. There is no comprehensive children's data base that gives a quick survey of 
what's available. Instead, working groups need to identify the people who develop, collect, 
and use data in iinpdrtant areas. Agency personnel, advocates, and planning — and research 
and development staff— as well as private-sector and non-profit analysts often will know 
exactly what the data are designed to measure, what limitations they have, how often they 
are updated, and whether they are likely to be a good fit with each result and indicator. 
Members of the working group will ofter\ include some of these people, who are a first good 
source of information. Even so, the sifting and sorting of data will be considerable. 

Set Parameters. A usdful strategy in managing data collection is to set some parameters 
for the process and then stick to them. Decide in advance on a reasonable number of results 
and indicators, next devise a work plan vyith ''drop dead" dates for finishing key parts of the 
work. The success of a time-frame strategy, however, depends on a realistic schedule, 
adequate staff support, a strong sense of commitment from participants, and an ability to 
keep moving forward while continuing to tie up loose ends in prior assignments. 

• The chairperson of Georgia's Task Force-on Accountability established expectations 
early on: The group was charged with developing a framework that had no more 
than 25 indicators for the five results already adopted by the Policy Council. A 
minimum number of meetings were anticipated and a completion date was 
assigned. The work was accomplished over a six-month period with five Task Force 
meetings and behind-the-scenes work by Georgia Policy Council staff. Based on a 
. work plan developed by the staff and approved by the Task Force, the process 
moved through several steps: 1) setting the criteria for selecting indicators; 2) 

developing a working list based on input from a literature review and talking with 
experts; 3) prioritizing indicators; 4) seeking community input; and 5) revising the 
indicators. As one staff participant put it, "We knew that we could have continued 
this work indefinitely, but we didn't have the luxury of taking two or three years. 
We took our best shot, knowing that data bases are always changing and that we 
would be able to come back and make improvements." 

Use Simple Decision Tools. Sorting through reams of data and keeping track of 
preferences in an. iterative process are exacting and time-consuming. Indicators are 
frequently discussed more than once, as new data are identified, then revisited with respect 
to an altogether different result. ' 

A simple decision matrix can be very helpful in keeping track of evaluations made 
along the way. A number of working groups have developed worksheets that lay out 
indicators for each result down one side of the page and their decision criteria across the top.. 
These formats visually display how each indicator stacks up on its own against each criterion, 
as well as in comparison to others. 

Another useful approach is to group indicators into levels or tiers according to the 
power of the data available to measure them. Twenty or more indicators can easily be listed 
for a given result. Yet not every indicator can be measured equally well. Therefore, three or 
four primary indicators should be selected to represent what a result "means" in measurable 
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and measured terms. But those that are not selected as fitting into this first tier should not be 
discarded. These indicators can be placed on a second list for use later in the process. A 
system of tiers ranks indicators according to theu* technical power, but keeps them in play 
until political decisions can be made. A third list of desired indicators where data needs to be 
developed or improved can serve as the ''data infirmary" and provide the basis for a data 
development agenda. Over time, group may add or move indicators from one list to another 
within this structure: v . , . 

• Hainpton, Virginia, developed a two-tier system and incorporated it graphically into . 
its decision matrix. They made "above the iine/below the hne'' distincti by 

. dividing the page horizontally into two sections. Indicators that did not perform well 
against key criteria were moved "below the line" but kept on the page. This simple 
technique allowed the group to inake evaluations but to keep indicators "in play" to 
consider later on. 

• , Vermont divided its indicators into three groups according to whether measures 1) 

were reliable and available at a county- wide or sub-county level across the state; 2) 
were reliable but riot consistently available; or 3) relied on qualitative measures 
collected and managed locally. They gave preference to first-tier, indicators, ^ but ’ 

. retained the others to help them think about "next generation" measures and how 
better data could be collected. 

Addressing the Lack of Useful Data 

A more challenging problem in finding a set of useful indicators stems from the nature of the 
data. First and foremost is that for many possible indicators, there simply are no data. An 
indicator like parent involvement, for example, considered by many to be essential to school 
success, is not something that states or even school districts have routinely been’ called on to , 
measure. ^ ^ 

In addition, many potential indicators are measured by inputs rather than by results. It 
is easier, for example, to find a statistic that gives the number of beds available for foster care 
placement than to find out how.many children who have passed through the system are now 
permanently living in stable families. 

Finally, many of the indicators for which we do have data rely on negative measures for 
positive results. Indicators of good health for children, for example, often include measures 
siidi as the iiumber of drug-exposed births, infant mortality rates, the frequency of teen 
births, and infant death rate, inong others. These measures keep us mindful of important 
problems that need fixing, but deflect our attention from creating the conditions in which 
children thrive. As one mernorable phrase puts it/ "problem-free is not fully developed. 

These limitations are grounded in an American tradition of social policy that says "if it's 
broken, we can fix it," rather than in an American entrepreneurial tradition that says "we c^ 
do it better." . They are perpetuated as well by agency-based data systems designed to meet 
quite narrow administrative and political needs.. Public information systems have never been, 
asked to create and track more useful measures of what states and localities think are 
important for children. And so they have not. 

What can be done? Letting the light of day shine on these gaps and inadequacies may 
be the best corrective. Until citizens and policy-makers begin to look, they won't have any 
way of knowing what data are being collected with taxpayer's money or take steps to buy , 
anything better. In one state initiative, a legislator involved in defining results and indicators 
expressed outrage when he was told that data for a specific measure of employment success 
were not being tracked. A state agency staff member put it simply: "Give us the money and 
we wiU." ’ ^ - 

In the meantime, jurisdictions can do at least three things: 1) recast their language and 
their thinking as much as possible to present a positive picture of children and families; 2) 
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develop new, positive indicators; and 3) develop near-term, low cost techruques to 
approximate desired data. Surveys of carefully selected sample groups can be used, for 
example, to approximate immunizations, parent involvement, and preschool attendance 
rates, as well as other indicators for which community-wide data are often unavailable. 

• Vermont believes in positive thinking. Participants in Lamoille County's People in 
Partnership coalition have tentatively agreed on three broad results: 1) young people 
who are successful in school; 2) young people who are successful in communities; 
and 3) young people who demonstrate caring behaviors. At the state level, 
Vermont's State Team for Children and -Families decided to give a more positive tone 
to ar\ indicator on many working lists: "adolescents avoiding high-risk behaviors." 
They changed it to "young people choosing healthy behaviors," using the term 
"young people" instead of "adolescent" to broaden the age group of concern. 

• Rochester's Change Coordinating Team developed a two-pronged strategy to 
measure substance abuse, an important indicator for which no good county-wide 
data were available. First, the school district, a key partner in Change, agreed to 
conduct a Youth Risk Behavior Survey on a bi-annual basis in all the high schools. 
Second, the Committee worked with state officials responsible for collecting regional 
data on substance abuse to make a special exception and to break out data specific to 
the Rochester City School District. 

Question #6: How Do We Resolve Conflict and Make Final Decisions? 

Even in the most efficient process, there will be intense give and take in the selection of 
results and indicators. This is because the process of moving from a working list to a 
consensus list is only partially technical. It is hard to exaggerate the extent to which 
developing a results-and-indicators framework is a political and often contentious 
undertaking. According to one participant in a state effort: "Everything is always an 

argument. Everyone fights over every word. What everyone wants to know is if there is an 
indictor that will affect them." 

People in cross-sector working groups and oversight bodies bring — by design — a wealth 
of values, cultural experience, religious perspective, education, and professional backgrounds 
to these conversations. Talking about children and families gets to the core of what most 
people believe in and hold most dear. "Data types" involved in the process introduce their 
own strong concerns about what is acceptable. It is not surprising that discussions about the 
"seemingly objective data" that we choose in order to help establish and measure our 
expectations for children ''generate a great deal of heated debate."^^ 

The challenge at this stage of the process is to negotiate a consensus list that everyone 
can support, not just accept because they've gotten tired of fighting. The process must not 
only come to a timely end, it must bring participants to agreement. A variety of steps can be 
taken to minimize conflict and negotiate a broadly owned list. 

Establishing Clear Expectations 

At the onset, there must be a clear distinction drawn between indicators of broad, 
community-wide results, and performance measures of specific agency or programmatic 
activities. Agency participants, in particular, need to understand that the indicators that they 
are selecting will not be used to evaluate their day-to-day operations. 

Continiiing efforts must also be made to help advocates, agency personnel, and others 
who bring special interests to the table to feel comfortable with the process. They need to 
understand that every individual issue cannot reasonably be spelled out in a single list of 
results and indicators. They must be assured, however, that a well-crafted list will be broad 
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enough to allow consideration of a wide variety of strategies ^d interventions in subsequent, 
stages of results-based budgeting. ■ ' 

Clarifying the Issues . " ■ 

When disputes arise, it is helpful to sort out the areas of disagreement and possible options 
and lay out the consequences of coining down on either side of the issues. Participants then, 
have to decide what reaUy counts, this kind of "heads up" strategy can be very useful in 
resolving disputes in which political and technical considerations are intertwined. 

• , Georgia's Policy Council, for example, was committed to its criterion of selecting oidy 

indicators for which there were statewide data. On the other'h^d, the commumty 
coUaboratives and other groups that it had asked for feedback strongly requested the 
inclusion of indicators for which there were no data, like youth substance abuse or 
accessible child care. The group had to balance the costs of using less-than-rigoro.us 
data in specific cases against the possible consequence that communities would not 
continue to support a process which disregarded issues that were so important to 
them. The Council decided that adopting a list ^at everyone could believe in was its 
first concern. It decided to include several indicators for which there were no data, 

and by so doing to call attention to the need for better measures in the future. 

Negotiating Agreements 

In some cases, making the terms of the dispute clear is not enough to resolve the matter. 
Frequently, negotiation is necessary to arrive at an agreement that has something in it for 
both sides— the oft-cited, "win-win solution. 'V The person best suited to broker a win-win 
arrangement is often someone who has had experience on both sides of the issue. 

' . • The Vermont team hit a roadblock that boiled down to one pivotal question: "To 

what level can you break down data and still be assured that it is statistically 
significant?" Some agency staff responsible for ensuring high-quality information 
did not want to report data for sub-county jurisdictions. Community representatives 

and others on the team, however, were concerned that relying only on state or county 
data obscured important within-county variations and thus made the data less useful 
at the local level. A member of the team who had formerly been a state health 
. department statistician, and vyho understood agency concerns negotiated a 
compromise. The team agreed to collect data at *e, school supervisory union, (school 
district) level because it was small enough to be considered "local" and large enough 
in most cases to be reliable. 

Developing a Series of Decision-making Strategies 

Another way to minimize controversy and make tough choices among indicators, is to 
establish one or. more decision-making strategies. Narrowing down an extensive vyprking list 
into a politically credible final product may require a variety of approaches. Consistently 
referring back to criteria established by the group to organize data and make determinations 
about their power is an essential first-order strategy. Another useful protocol may be to 
exclude any indicator which at least one participant, after reasonable discussion, stiU canriot 
accept. This ensures a base-line consensus. Rank-ordering is a tfurd strategy that can be 
useful in establishing priorities and narrowing down lists that are stiU too long. 

• Designing a rank-ordering strategy can pose its own difficulties. The first attempt of 

Iowa's Council on Human Investment to prioritize its proposed benchmarks did not 

deliver the information needed to do the job. Research polls were conducted asking 
lowans to evaluate the importance of each indicator on a scale from 1 to 5. When the 
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results were tabulated, most mdicators showed up as 3's — of moderate importance. 
No useful distinctions emerged. In a second effort, surveys asked people to rank the 
importance of each indicator relative to the rest of the list. 

• A "critical factors" analysis proved an essential ranking tool for Georgia's Policy 
Council. Even after its Task Force on Accountability had assessed its initial working 
list according to four selection criteria, more than 40 indicators remained. It devised 
a "critical factors analysis" to prioritize its indicators and bring the list down to no 
more than five for any one result. Since many of the indicators reflected problems 
that Georgia wished to address. Task Force members were asked to rardc each 
indicator according to 1) the magnitude of the problem and the extent to which it 
affects the state budget; 2) the seriousness of its consequences and the costs of letting 
it go untreated; and 3) the feasibility of correcting the problem given, the^current state 
of technology, knowledge, and resources. Two rounds of ratings were used. In the 
first round, members individually rated each indicator. Then the list was aggregated 
and discussed by everyone. A second round of ratings established the group's final 
agreement 

Question #7: How Can We Use the Process to Build Support with Diverse Constituencies? 
An effective results-and-indicators framework is politically credible. It gains legitimacy 
when diverse groups feel that it captures the conditions essential for* the well-being of 
children and families, and points to areas where public and private action are most needed. 
As such, it has the support of. overlapping constituencies, including those who hold major 
resources and those who vote. 

The process of building a constituency begins in the earliest stages of moving towcird 
results-based accountability. This guide suggests that in deciding who should be in charge of 
such an effort jurisdictions should look to — or create — ^an oversight body that is supported by 
both the executive and legislative branches of government, and includes the participation of 
elected officials from both major parties. Equally important is the inclusion of a broad cross- 
section of corporate, civic, and religious leaders with roots in a range of cultural, racial, and 
geographic communities. 

A second opportunity for constituency building comes as initiatives make decisions 
about the composition of working groups. Whether they be subcommittees of public-private 
oversight bodies or cross-agency working teams, working groups would be wise to 
incorporate the views and perspectives of key constituencies. 

A third opportunity for constituency building comes as state, county, dty, and 
community initiatives make decisions about how to engage the public in, the process of 
selecting results and indicators. Some initiatives directly involved thousands of residents in 
establishing statewide results, collecting information through town meetings, polls, and 
surveys. Others began with lists of results and indicators, narrowed them down, and then 
asked for public comments from both the general public at open sessions or from community 
coUaboratives. ^ 

Building a constituency begins By taking advantage of . these three opportunities. But it 
should riot end there: Adopting a political credible list of results and indicators is an ongoing 
process. It establishes new relationships and a set of attitudes, expectations and 
commitments about our children, and how we use our resources, that grow over time. 
Developing a constituency that shares these expectations and commitments requires 
continuing attention to: 1) building bipartisan support; 2) establishing links with existing 
data collection efforts and key community organizations; and 3) encouraging localities to use 
and develop their versions of results and indicators. 
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Building Bipartisan Support 

Short-term election cycles frequently haVe a destabilizing effect on long-term agendas. Term 
limits compoimd this problem. Even initiatives that are weU grounded politically must spend 
time cultivating the interest and support of newly elected and returning officials on both 
sides of the aisle. 

Incoming policy-makers in state government, county coimdls, or city halls may not be 
aware of efforts already -imder way to move toward a results-based accountability system. In 
other cases, there may be awareness arid even interest, but other issues may be higher on 
policy-makers' lists of priorities. Steady communication is needed to bring poUcy^makers of 
both parties "up to speed," keep them informed, and help them see the value of becoming 
involved. 

Nor should did friends'be taken for granted. Retuiming supporters provide the stability 
and institutional memory that make it easier for new particip^ts to get on board. 
Continuing public appreciation of the work of old supporters and understanding the limits 
on their, time, while frequently seeking their advice, can play a part in broadening and 
deepening their participation. Adopting an initial set of results and indicators only begins the 
shift to results. Policy-makers weU versed in the strategy and tactics of the political process 

^d the minutiae of the budget process are essential to push the process forward. 

• In Iowa, members of the Council on Human Investment engaged the interest of 
legislators in results-based accoimtability by appealing to -their self-interest. . They led 
off with a simple question: "What land of stories do you want to take home: how 
much money your programs cost, or what they accomplished?' Once elected- 
officials understand that a results-based system can help them communicate more, 
honestly and effectively with their constituents, then they are more likely to support 
■ the adoption of a community-wide set of indicators and to use them in requesting 
data, evaluating performance, and approvmg budgets. 



Establishing links with Existing Data Collection Efforts and Key Community 
Organizations' 

Any jurisdiction attempting to adopt a politically credible list of .indicators should seek out 
the help and participation of other groups on whose work they might build. These projects 
and the people involved with them caii budld links to a variety of constituency groups. 

Many of the institutions sponsoring these activities are highly regarded fixtures in their 
communities. Others have acquired legitimacy based on the value of the information they 
deliver. United Ways, for example, and other civic orgaiiizations have long been involved in 
conducting environmental scar\s and developing single-point-in-time snapshots of where a 
community stands on given issues.. National initiatives like Goals 2000 and their state and 
local coimterparts have set expectahons for educational attainment. Coming even closer to a 
results-and-indicators framework is the work of the’ natioriail Kids Coimt project,' which 
compiles annual state-by-state profiles of the health; economic, educational, and social 
condition of children based on the best available data. In many cases, there are also state- 
level Kids Coimt projects that are designed to devdop local indicators and strategies to 
improve results for the most vulnerable children. 

States ^d commuriities undertaking a comprehensive strategy to move tovyard results- 
based accountability should make conscious efforts to build on and ensure compatibiUty with 
the work already under way in their jurisdictions and to speak directly to the constituencies 
that they have already developed. Every effort should be made not to alienate these potential 
partners by ignoring their existence or underestimating their contributions. Conversely, 
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initiatives should have a clear sense of purpose and be ready to negotiate compromises when , 
questions arise about how to incorporate prior work. 

• Some jurisdictions, including Georgia and Rochester, regarded compatibility with 
existing data measures as a primary criterion for selecting indicators. Georgia's 
protocol required consideration of work conducted at the both the state and local 
level by Kids Count; the Council for School Performance; the Savannah Youth 
Futures Authority, a local community collaborative; and the Governor's Council for 
Economic Development. 

• In Rochester, the United Way is one of the Change Collaborative's four permanent 
members, so United Way data were a key part of the Coordinating Council's data 
collection process. Deliberate efforts, however, were made to synchronize work with 
the New York State Kids Count project being developed at the same time, in order to 
ensure a compatible set of indicators. The Coordinating Council also drew on a 
recent "state of the city" report developed by a private research and analysis outfit. 
This firm became an important source of technical assistance and was later asked to 
establish historical baselines and track progress at the city and neighborhood levels. . 

• In Minnesota, inclusion of other efforts led to a useful discussion about the balance 
that an effective framework should strike among social, health, educational, 
economic, and other indicators. In one state agency, work had already progressed 
quite far on an extensive set of economic indicators. There was considerable feeling 
^at they should all be included. The question became: "How and to what extent 
should these indicators become part of a comprehensive framework on statewide 
well-being?" Minnesota Planning, the state agency responsible for developing the 
list, then consulted with its Advisory Committee. Members negotiated a solution in 
which a selection of the most important indicators were included in the body of the 
final report. A complete discussion of a statewide "economic blueprint" was 
included as an appendix. . 

Encouraging Localities to Use and Develop Their Versions of Results and Indicators 
The most important constituencies — and those that need to be most carefully nurtured— are 
communities and neighborhoods themselves. Improvement in the conditions of children and 
families starts and ends at the local level. That is where children and families live, and where 
solutions must take hold. According to one participant at the state level: "Central to all this 
work on results and indicators is a radical belief that commimities can organize themselves to 
change direction." The Los Angeles County Children's Plaiming Council put it simply in a 
strategic planning document: "Wonderful, difficult, and — in some cases — astonishing work 
is happening in communities. .. . This work must be supported, continued and expanded." 

A successful initiative to improve results for children and families is a community effort 
that must take place at both the state and local level. Localities are where priorities must be 
set and solutions owned and implemented. State-initiated efforts need to consciously 
encourage localities to adopt or adapt statewide measures and to incorporate these measures 
in their own strategic' planning. States have the resources and bear the full burden of 
accountability; they ako have the capacity to resolve policy barriers that may impede 
progress. Wherever they are initiated, results-and-indicators fraiheworks can provide the 
glue to create — not mandate — equitable state and local partnerships. 

• Georgia is one of a growing number of states that are developing formal mechanisms 
for encouraging localities to develop ^d use results-and-indicators frameworks.^® 
Legislation authorizing the Georgia Policy Council for Children — the oversight body 
charged with establishing results and indicators — also called for the creation of local 
community partnerships. Communities with a strong track record on a set of 
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readiness criteria have agreed to work on a core set of results based on a statewide 
framework. They will be. expected to develop a strategic plan to achieve core results 
in return for more flexibility in pooling resources across systems. Joiiit efforts are 
under way to clarify mutual expectations about state and local roles in the 
partnership and to make it easier for communities to collect and use state-level 
information. 

, Whether or not formal state and local partnerships are in plaice, it is essential that residts 
and. indicators framed at higher levels be perceived as useful in local decision-making. States 
and localities have i^perimented with a variety of ways*tb do this. Some of these strategies 
have been discussed in different contexts elsewhere in this report. 

• Listening carefully to community and neighborhood feedback about what indicators 
should be selected and acknowledging and/or making accommodations to include 
them. Doing so sends a powerful signal to localities that adopting this list is not just 

, another bureaucratic exercise. 

• Presenting data that paints an accurate picture of local conditions. Whenever 
possible, data should be disaggregated to show important variations by age, racial 
group, and other dimensions that .may be hidden in. unified measurements. 

• Taking steps to assist local irutiatives with their data needs, whether local indicators 
are on the state's list or riot. 

• Making infohnation easily accessible.. A number of jurisdictions are using a variety 

of formats to build awareness of results and indicators, mcluding: "hard-copy" 
documents available ' for free; short summaries of "rack reports" available at 
supermarket checkout lines and bulletin boards in a variety of community locations; 
and, increasingly, information in down-loadable form on the Internet. Lamoille 
County, . Vermont, is experimenting with a series of posters that list key indicators 
along with illustrations and tag lines that make them immediately hit home. Oh one 
poster that tracks county* rates of child abuse and neglect over the past ten years, the 
caption reads: "Being a child. should not hurt." The Geor^a Policy Council also 
created arid widely distributed posters identifying and highlighting results and 
benchmarks. . ’ . ' .. 

• Provide training and user-friendly backup materials to help people see what can be 

done with the data and to feel comfortable using it. In Vermont, a traiiung module 
based on a hypothetical community case study has been developed , to help people 
use data for decision making. So far, this capacity-building has been limited to 
working with local health department directors so that they, in turn, can help 
community groups use the data. In the future, staff hope to more systematically 
"train trainers" in interested communities. . 

Question .#8: What Can We Do to Begin Using Results and Indicators in the Budget . 
Process? , 

This is the question that brings the process full circle: Why develop a results-tod-indicators 
framework in the first place? For two reasons: first, to set a clear direction for jurisdictions 
committed to making a difference . in the lives of children and families; and, second, to 
provide a context in which to inake tough choices about how to spend limited public and 
private dollars. Initiatives, however, can become so bogged down in the minutiae of creating 
a framework that this becomes an end in itself — the list becomes more important than the 
decision-making that it is intended to affect. 
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Some states and localities, however, are making notable progress. No jurisdiction is yet 
at the point where results routinely and consistently drive how public and not-for-profit 
budgets are managed and . services are designed, but several strategies appear promising. 
Experience suggests that effective initiatives begin building connections between the 
framework process and the budget process at the earliest stages of their work. They develop 
and apply progressively more sophisticated tools, and they implement specific, often 
incremental, strategies for operationalizing the link between results and budgets. 

Forge Connections Early 

Results and indicators should not be developed in a vacuum. Parallel developments in state 
and local govermnents, as well as in, other major institutions, regarding allocations and 
budgeting need to be taken.into account. In a growing number of jurisdictions, for example, 
agencies are being required to develop performance measures. Irutiatives need to find ways 
to ensure that state players are aware of statewide results and indicators and that they see the 
relationship between these measures and their own agency requirements. Making these 
connections can begin with issues as basic as who's involved — ^by including budget staff and 
the financial community from the onset. . 

• . In Georgia, 'members of the Results Accountability Task Force included a former 

director of the Governor's Budget Office. Having been closely involved with earlier 
efforts to implement performance-based budgeting, he helped the group grasp the 
difference between agency-level performance measures and statewide results. The 
Task Force then worked with people responsible for developing state budget 
instructions in order to agree on common definitions and to see how the two 
processes could fit together. 

Setting expectations that results will be used as guidelines . in developing agency 
spending plans is a necessary but insufficient strategy for tying results to budgets. Budget 
appropriations should eventually be approved on the basis of how directly they respond to 
key indicators. Other papers in this series wiQ explore how this cm be done. But even in the 
early stages of moving toward results-based budgeting discussed in this paper, it is clear that 
mal^g the link between results and budgets requires strong leadership and explicit budget 
tools. 

• Leadership in Multnomah County, Oregon, for example, has used the cinnual county 
budget to formalize the county's commitment to making progress on specific, 
"urgent" indicators. The county budget document has been used, to show the 
relationship between results, indicators, strategies, and the county's current and 
future funding agenda. Although these relationships are not yet deeply embedded in 
the county budget development process, forging these early links strerigthens the 
county's ability to make the longer-term transition to results-based budgeting. 

Develop and Refine Indicator Tools 

A simple results-and-indicators list— no matter how politically credible — is not powerful 
enough to drive the budget process by itself. Conscious efforts must be made to use the list 
as the basis for developing a set of tools that are strong enough to reshape dec^ion-making. 
In. its first and simplest iteration, a list of results and indicators can be used to inform an 
annual status report — a single-p6int-in-time description of commuiuty well-being on selected 
measures. Many initiatives are working at this level. At a second level of work, baseline 
measures and historical trends can be added to put current conditions in perspective and to 
develop the framework's ability to forecast trends if current conditions continue. At a third 
level of work, the framework can be used to track progress against baseline measures on a 
frequent basis. This guide concentrates on the first level of application. It is importmt. 



THE FINANCE PROJECT 






however, that jurisdictions recognize the importance of systematically refining their efforts 
and creatively using, rather than statically revering, the frameworks that they have created. 

Work Incrementally 

In most cases, tangible links between results and budgets are best forged incrementally. 
Some jurisdictions may have the political will and management capability to begin shifting 
their eritire budget to results-based accountability all at once. Many others may wish to begin 
by earmarking a portion of available funding to a particular set of indicators, or to channel 
funds to Ipcalities'that agree to work on priority issues. 

• When explicit strategies are used to tie indicators to funding, there can often be 
pronounced state action. In one state budget cycle in Oregon, for example, a pool of 
some $100 million was created by across-the-board budget cuts initiated by executive 
action. The legislature then reallocated funds to agencies whose budget proposak 
addressed "urgent" indicators — those considered to be the state's most pressing 
problems. A notable shift in priorities occurred as a result. 

• - In another focused effort, Oregon created a Commission on Childr^ and Families to 

coordinate state-level child and family policy and local service delivery around 11 
state-specified indicators. It disbursed approximately one percent of the total state 
human resource budget in discretionary funds to local commissions, in each of the 
state's 36 counties. Local commissions. set their owri priorities. They can use state 
funds however they decide, as long as their local service system plans emphasize 
wellness md are aligned with one or more of the 11 child and family indicators that 
they select.’^ 

• Leadership originating at the local level can also push the envelope toward results- 
based decision-making. In Rochester, New York, partners in ^e Change 
Collaborative signed a joint agreement to begin using; results and indicators to 
allocate resources in their respective systems. Instead of funding agencies,' the 
United Way is conducting a pilot project designed to fund results. First, it created a 
'pool of money from performance-related and across-the-board- budget- cuts. Thei> it 
made additional funds available to high-performing agencies to apply toward 
activities focused on selected results and indicators. County government, another 
partner in the Change Collaborative, is also starting to take action. Its Youth Bureau 
has made focused attention on specific community indicators a prerequisite for 
awarding contracts. 

Question #9: How Can We Make Sure that the Process Keeps Moving Forward? 

States and localities working to adopt a list of results and indicators need to remember that 
an effective framework is intended to be revised and updated frequently. .While a politically 
credible list of results and indicators must be a sensitive reflection of essential community 
values, a first-round effort does not need to be a perfect informational tool. And, given the 
imperfect state of ^ knowledge and available data — as well as changing community, 
circumstances— it won't be. An ongoing process is aimed at developing progressively more 
accurate information embedded in increasingly more useful tools. _ 

Iii many cases, for example, scorecards that report data on results and indicators are 
updated on an annual or biannual basis. This update by itself, however, is not enough to 
ensure a dynamic and self-correcting process. Some indicator reports track the same 
measures over several years, an approach taken to show tiends over time across the s^e 
nieasures. In some instances, however, reliance on an unchanging set of measures may signal 
a shift from "process to project" and work that has gone on "autopilot." UiUess the results 
and indicators used to create the scorecard are arid continue to be the very best reflection of 
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community desires and the most useful indicators by which to measure them, consistency is 
probably no virtue. 

As the state and local illustrations in this gmde suggest, initiatives c^ do several things 
to ensure a constantly evolving and dynamic process. First, working groups charged with 
revising results and indicators are infused from time to time with new participants who bring 
fresh ideas, questions, and perspective. Second, effective initiatives continue to refine their 
understanding of how strategic planning and the budget process can be more closely linked. 
Then they apply that knowledge. Third, initiatives continue to solicit input from groups at all 
levels who use results. They are constantly looking for ways to measure what communities 
consider important. They think in terms of "next generation" indicators and how to measure 
them, as well as looking for better data to measure indicators already in place. Fourth, 
initiatives are not afraid to ask themselves hard questions and to open theu process to public 
scrutiny and improvement. An impressive example of this last strategy is described below. 

• At the current Governor's request, Oregon recently conducted a comprehensive 
review of its results-and-indicators framework, based on public testimony, research 
by staff, and survey information from state agency directors and administrators. The 
purpose was to recommend steps that should be taken to stren^en the state's ability 
to use results-based standards. Several subcommittees of the citizen-based Oregon 
Progress Board were asked to answer a series of key questions, including: 1) How 
well have indicators been integrated into state policy frameworks? 2) Who is using 
the indicators, and how can more users be encouraged? 3) Are the right numbers and 
kinds of indicators being measured, how can they be presented more effectively? 

• The findings pointed to specific steps needed to strengthen Oregon's ability to use 
indicators as a decision-making topi.” As an example of Oregon's continuous 
improvement, action was taken and reflected in a revamped version of "Oregon 
Shines," the state's strategic plan. The revision reduced the number of benchmarks 
from 270 to 92 and focused state effort on. three top goals: creating and sustaining 
high-quality jobs, caring communities, and healthy surroundings. This reconstituted 
state plan and the benchmarks used to monitor progress in key areas have been 
embraced by some of the state's most powerful public officials — and greatly improve 
the chances that Oregon will increasingly use results and indicators in budget and 
policy decisions. As one former, legislative skeptic declared, "This is a plan for the 
future of the state." 

CONCLUSION: 

LESSONS LEARNED 

This guide ends where it began, with a simple question: How do commimities, along with 
their governments, make sure that they are providing the basic conditions that all children 
need, not just to survive, but to flourish? Implementing that simple idea, given a set of 
institutions ^at have not been well designed to answer the question, is a complex 
undertaking. It depends on a day-by-day process of reinvention. The following cautionary 
advice, distilled from the experience of states and locaUties that are already weU down the 
road, may be helpful as new jurisdictions decide to move forward. 

• Don't Expect to Get It Right the First Time. Remember that an effective framework 
is both a product and part of an ongoing, strategic process. In selecting results ^d 
indicators, aim for the best approximation of "perfect" possible in the time allotted 
and then view feedback about imperfections as valuable field research, Maintaining 
a long-term view of the process helps participants to pace themselves and keep a 
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positive focus. Recognizing in advance that there will be continuous opportunities 
for improvement and that other tools to help decide on particular programs and 
strategies will be developed, also makes it easier to negotiate broad agreement 
among multiple special interests. . 

. . • Watch but for Techies! Crafting a resiilts-and-indicators list is primarily a political 
problem. Techiucal expertise is essential in collecting and interpreting data and 
"evaluating the extent to which data for given indicators are strong, reliable, and 
accurate. But the indicators finally selected must also be true to community values. 

. Unswerving allegiance to technical criteria may make for the most statisticaUy sound 
framework^ but one that falls short as a powerful lever for community action. When . 
si^fficiently strong data to measure indicators that communities consider of great 
importance are not available, strategic decisions about how to proceed should riot be 
made on technical grounds alone. 

• Don't Oversell. A results-and-indicators framework is only one tool in an overall 
strategic plan to improve conditions for children and families — not' a panacea. Its 
existence won't automatically throw . a switch that changes how people think, ' 
institutions operate, and budget decisions are made. Be clear about what results and 

. . indicators carinot be expected to accomplish on their own, and. then spell out how 

they can help. What they can do is to provide a directional compass to help 
communities keiep focused on where they want to go; serve' as an anchor with which 
to ground their work and develop other decision making tools; and act as inagriets 
that encourage collaboration among diverse interest groups and sectors of the 
community. 

• Balance Input with Manageability; Aim for a short, positive list of fundameritals, 
not an unattainable wish list or a checklist of every conceivable problem. Enough . 
work has been done on results for children and families so that communities do not 
have to starhfrom scratch to build a list that is -^uruquely their own. Begin with a 
workirig list; decide on a limited number of indicators and a completion date; and 
solicit focused comments from community groups who are respected in the 
community* and familiar with the issues. 

• Get Connected. The heart of results-based accountability lies in tying results to 
budgets. Initiatives must link the broad range.of efforts focused on improving the 
well-being of children at the local level and then connect them to the support and 
resources available only through state-level action. A dynamic process provides for 
local variations. It develops specific strategies for linking statewide results to 
community-based decisions about what results are rhost important and how progress 
should be made. 

Finally, .people heed to get connected with their peers around the country who are 
working hard to make sure that all our children have a fair chance to succeed. As the stories 
in tte report suggest, there are . growing numbers of people in states and localities facing 
similar* challenges who can learn from and help each other. We hope that you continue the 
conversations begun in this document, expand them, and share your knowledge with others. 
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Principles for Outcome Measurement for Family and Children's Services 

Los Angeles County , V 

1. Outcomes and indicators should be practical and resultSToriented> clearly important to 
the well-being of children, and stated in terms that are understandable to the public. 
They should reflect the well-being of the. whole child, rather than focusing on the parts 
served by specific service systems. 

2. The overall outcomes sought should be expressed as poritive expressions of child well- 
being, rather than the absence of negative conditions (i.e., good health rather thai\ decreased 
illness). However, many of the indicators that measure those outcomes will be phrased in 
the negative because that is how data is currently collected. 

3. Since no one indicator captures the full dimensions of the outcomes sought, each outcome 
should be measured by a set of indicators chosen from ■ the most valid cind reliable data 
available. 

4. Indicators should be selected to reflect the overall state of oixr children, not the state of 
the service delivery system, although implications for the improvement of the current 
system of services should be derived frorh the regular collection cind analysis of service 
delivery data. Indicators should, where possible, reflect the outcomes of seryices for 
families and childrm, and not just the existence of services. 

5. Initial efforts should focus on a strategic set of outcomes and indicators that reflect 
concerns^ shared by the entire community, including policy-makers, service providers, 

' and families. Efforts should begin with a limited number of outcomes arid indicators that 
focus on child well-being, with the understanding that in subsequent years, indicators 
that reflect the well-being of families and communities may als6 be added. . 

6. The process of developing appropriate and practical outcome measures that accurately 
reflect the state of the county's children will be an evolutionary one, from which there is 
much to learn. Perhaps one of the most importmt steps is the clar^cation of the cultural 
and value foundations that underlie the process; the selection of outcomes and indicators 
that, reflect goals shared by all groups is essential if the product is to be a meaningful 
picture of the state of the county's children. 
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About The Finance Project 

The Finance Project is a national initiative to improve the effectiveness; 
efficiency, and equity of public-, and private-sector financing for education, other 
children's services, and community building and development. With leadership and 
support from a consortium of private foundations. The Finance Project was established 
in 1994 as an independent, non-profit organization. It undertakes an ambitious array of 
policy research and development activities, policy maker forums and public education 
activities, and support and technical assistance activities. 

The work of The Finance Project is aimed at increasing knowledge and 
strengthening the capability of communities, states, the federal government, non- 
governmental entities, and the private sector to implement promising strategies for '. 
generating necessary fiscal resources and improving the return on investments in ' ■ 
children and their families. Its activities are intended to: 

• .Examine the ways in which governrhents at all levels, and the private sector, . 

finance education and other supports and services for children (age 0-18) 
and their families: . , 

• Identify and highlight structural and regulatory barriers that impede the 
effectiyeness of programs, institutions, and services, as weli^as other public- 
investments, aimed at promoting childr;en's grovvth and development: 

• Outline the characteristics of financing strategies and related structural and 
administrative arrangements that support improvements in education, other ' 

, ^ children's services, and community building and development: 

• Identify prornising approaches for implementing these financing strategies at 

the federal, state, and local levels and assess, their costs, benefits, and 
feasibility: ; 

Highlight the necessary steps and cost requirements .of converting to new, 
financing strategies: and 

Strengthen intellectual, technical, and political capability to initiate major long- 
term reform and restructuring of financing systems, as vyell as interim steps to 
overcome inefficiencies and inequities within current systems. . 

The Finance Project's work is organized within three broad areas. Activities in 
each area build upon and inform the others: 

I. Review and analysis of federal, state, and local financing strategies for 
education, other children's services, and community buijding and development. 

Through a series of working papers and other studies. The Finance Project examines 
key issues and trends in federal, state, and local fiscal capacity and public expenditures 




on behalf of children and their families and assesses strategies and challenges for 
generating fiscal resources, planning and budgeting public expenditures, and measuring 
the impact of public investments. It also examines the implications of relevant statutes, 
regulatory provisions, and judicial decisions for patterns of expenditure. Approximately 
40 studies have been conducted and published to date. • 

II. Development of Ideas, options, and policy tools. 

The Finance Project organizes and convenes interdisciplinary working groups to 
serve as development and design teams for policy-relevant ideas, options, and tools for 
financing reforms. Additionally, it initiates and participates in collaborative efforts to 
mobilize the intellectual and technical resources needed to bring about positive change. 

The working groups bring together individuals and ideas from many domains to 
identify and assess alternative approaches for generating necessary fiscal resources 
and improving the return on public investments; to develop options for more effective 
fiscal policies and streamlined adrhinistrative practices; and to design prototypes, 
models, tools, and materials to support their implementation in . the years ahead. Three 
working groups are developing tools and options in the following areas; 

• Strategies for Generating Revenue for Education and Other Children's 
Services; 

• Results-based Planning, Budgeting, and Accountability Systems; and 

• Financing Comprehensive, Community-based Support Systems. 

Additional working groups formed as needed. 

Led by the Finance Project, the Collaborative Initiative to Improve Financing for 
Young Children and Their Families, a group of national organizations, as well as 
selected state and local leaders who are involved in efforts to improve early childhood 
financing. They have undertaken a series of related activities to build a. responsive 
support and technical assistance capacity to help “reform-ready” states and communities 
to advance their agendas; 

• Clarify issues that are unique to financing community supports and services 
for young children and their families; 

• Identify and assess promising reform strategies; 

• Develop and distribute relevant policy tools. 

III. Promotion of improvements in financing systems and strategies. • 

The Finance Project reaches out to an array of audiences that are critical players 
in the change process — including federal, state, and local Officials; educators, human 
service providers, and community organizers in non-profit and quasi-gov'ernmental 
organizations; business and union leaders; academic scholars and foundation 
executives; and the media. The- Finance Project has created a valuable central 
intelligence capability on financing issues and strategies, as well as a rich support and 
technical assistance resource for "reform ready" states and communities. Drawing on 
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, the knowledge and tools that are being produced, the project's staff, consultants, and 
working group members work directly with state and community leaders in several vyays: 

• Facilitating direct connections with peers and experts who have relevant 
knowledge, expertise, and experience. 

• Providing on-site learning -opportunities. , 

. • Producing tailored "how to" materials. 

■ • Creatirig learning clusters.' 

In September 1997, The Finance Project was awarded a contract from the Child , 
Care Bureau of the U.S. Department of Health and Human Services to work with the 
Families and Work Ihstitute and the National Governor's Association to provide 
information and technical assistance to state child care administrators as they . . 
work with businesses, philanthropic organizations, and other groups to. build and sustain 
partnerships. Initial activities will , 

• Identify and track existing practices for promoting public-private partnerships 
at both the ’state and community levels; 

• Develop a series of written technical assistance materials to guide the 
formation and implementation of successful public-private partnerships; and 

• Provide peripdic updates to state child care grantees about information 
collected and produced. 

The Finance Project extends the work pf many other organizations and blue- ■ 
ribbon groups that have presented bold. agendas for imjDroving supports and services for 
children and families. It is creating the vision fora rnore rational approach to generating 
and investing resources in education, other supports and services for children arid 
families, and communities. It is developing ideas, options, and policy tools to actively 
foster positive change through broad-based systemic reform, as well as through more 
incremental steps to improve the effectiveness, efficiency, and equity of current systems. 
It also provides support and technical assistance to “reform ready” states, communities,, 
and initiatives engaged in efforts to align their financing systems with their policy and 
program reform agendas. 

For more information about The Finance Projecfand its.activities, please contact: 

Cheryl D. Hayes, Executive Director 
, , . The Finance Project . ‘ 

1000 Vermont Avenue, NW, Suite 600 
■ Washington, DC 20005 ; 

202/628-4200 

- 202/628-4205 (Fax) ' : . ' 

, . infb@financeproiect.org (E-mail) 
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RESOURCES AVAILABLE FROM THE FINANCE PROJECT’S WORKING PAPERS SERIES 

Federal Financing Issues and Options 

Financing Services for Young Children and Their Families: Meeting the Challenges of Welfare 

Reform by Cheryl D. Hayes (March 1997) 

■ Federal Tax Reform: A Family Perspective by Michael J. McIntyre and C. Eugene Steuerle 
[Report and Executive Sununary] (July 1996) *Please note there is a $10.00 charge for this 
publication. ' ... 

The Budget Enforcement Act: Implications for Children and Families by Karen Baehler 

(November 1995) . . 

Dollars and Sense: Diverse Perspectives on Block Grants and the Perso7tal Responsibility Act (Joint 

publication of The Finance Project and the American Youth Policy Forum and The Policy 
Exchange of the Institute for Educational Leadership) (September 1995) 

' Rethinking Block Grants: Toward Improved Intergovernmental Financing for Education and Other 
Children's Services by Cheryl D. Hayes, with assistance from Anna E. Danegger (April 1995) 

' Reform Options for the Intergovemrriental Funding System: Decategorization Policy Issues, by Sid 
Gardner (December 1994) 



State Financing Issues and Optioris 

___ Money Matters: A Guide to Financing Quality Education and Other Children's Services (January 
1997) *Please note there is a $20.00 charge for this publication. 

The Ejfects of Economic and Demographic Changes on State and Local Budgets by Sally Wallace 

(December 1995) 

Issues and Challenges in State and Local Finance by Therese J. McGuire (November 1995) 

Toward State Tax Reform: Lessons From State Tax Studies by, Therese J. McGuire and Jessica E. 

Rio (November 1995) . ' . 

, Legal Issues and Constraints Affecting Finance Reform for Education and Related Services by 

Thomas Triplett (November 1995) 

State Investments in Education and. Other Children's Services: The Fiscal Challenges Ahead by 

Martin E. Orland and Carol E. Cohen (November 1995) 

State Investments in Education and Other Children's Services: Fiscal Profiles of the 50 States by 

Steven D. Gold, Deborah A. EUwood, Elizabeth I. Davis, David S. Liebschutz, Sarah Ritchie, 
Martin E. Orland, and Carol E. Cohen (October 1995) 

. State Investments in Education and Other Children's Services: Case Studies of Financing 

Innovations by Ira M. Cutler, Alexandra Tan, and Laura Downs (September 1995) 



Spending and Revenue for Children's Programs by Steven D. Gold Deborah A. Ellwood 
(December 1994) ‘ , 



Local Financing Issues and Options 

_* Money Matters: A Guide to Financing Quality Education and Other Children's Services (January 

1997) *^Please note there is a $20.00 charge for this publication. 

The Property Tax in the 21st Century by Hal Hovey (May 1996) 

Issues and Challenges in State and Local Finance by Therese J. McGuire (November 1995) 

Forthcoming * 

Tax Incentives for Community Development hyPaulPTyde,]T. {Summer 199S) 



Financing Comprehensive, Community-based Supports and Services 

■ ' . Financing Strategies to Support Comprehensive, Comniunity-based Services for Children and . 

Families by Mary M. O'Brien, National Child Welfare Resource Center for Orgeinizational 
^ Improvement (March .1997) ‘ ■ : 

Building Strong Corrimunities: Crafting a Legislative Foundation (December 1996) ’^Please note , 

there is a $20.00 charge for this publication. ' . 

Building Cornprehensive, Community-based Support Systems for Children & Families: A' Review of 
'Legislative Examples by Thomas YJoods {DecemheT l996) 

- • Beyond Decategorization: Defining Barriers and Potential Solutions to Creating Effective 

Comprehensive, Community-based Support Systems for Children and Families by Martin E. 

Orland and Ellen Foley (April 1996) . 

^ Conceptualizing the Costs of Comprehensive, Community-based Support Systems for Children by 

Jennifer King Rice (November 1995) 

, Creating More Comprehensive, Community-based Support Systems: The Critical Role of Finance by 
Martin E. Orland, Aima E. Danegger and Ellen Foley (November 1995) 

Compendium of Comprehensive, Community-based Initiatives: A Look at Costs, Benefits, and ^ 

\ Financing Strategies by Cheryl D. Hayes, EUse Lipoff, and Anna E. Danegger (July 1995) 

____ The Role of Finance Reform in Comprehensive Service Initiatives by Ira M. Cutler (December 
1994) ' ' . . \ ^ 



Results-based Planning, Budgeting, Management and Accountability Issues 

I ■ • . . . 

- A Guide to Developing and Using Performance Measures by Mark Friedman (May 1997) 

• A Guide to Results and Indicators by AiehaMelavihe (May 1997) 

. A Strategy Map for Results-based Budgeting: Moving from Theory to Practice by Mark Friedman 

(September 1996) , - 
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Forthcoming 

Results-based Planning, Budgeting, Managemeyit, and Accountability Strategies: An Annotated 
Bibliography by Anna E. Danegger (Summer 1998) 

A Guide to Developing Child and Family Budgets by Mark Friedman and Anna E. Danegger 
(Summer 1998) 

Privatization, Contracting, and Reform of Child arid Family Social Services by Sheila B. 
Kamerman and Alfred J. Kahn (Summer 1998) 

Financing Early Childhood Supports and Services 

Revenue Generation in the Wake of Welfare Reform: Summary of The Pilot Learning Cluster on 

Early Childhood Finance (August 1997) 

Financing Services for Young Children and Their Families: Meeting the Challenges of Welfare 

Reform by Cheryl D. Hayes (March 1997) 

Forthcoming . ' 

Issues, Challenges, and a Proposed Research Agenda for Early Childhood Financing (Summerl998) 

School Finance Issues 

Securing Equal Educational Opportunities: Past Efforts and the Challenges Ahead by Alexandra 

Tan and Martin E. Orland (February 1995) 

School Finance Litigation: A Review of Key Cases by Dore Van Slyke, Alexandra Tan and 

Martin E. Orland, with assistance from Anna E. Danegger (December 1994) 



Working papers produced by The Finance Project cost $7.50. However, Federal Tax Reform: A Family 
Perspective [Report and Executive Summary] is $10.00. In addition. Money Matters: A Guide to Financing 
Quality Education and Other Children's Services and Building Strong Communities: Crafting a Legislative 
Foundation cost $20.00 each. Please indicate the quantity desired next to the publications you would like to 
receive and mail this form, along with your payment, to: . 

The Finance Project 
1000 Vermont>Avenue, NW, Suite 600 
Washington, DC 20005 
202/628-4200 

Name:, - ^ Title: 

« '■ > . . 

Affiliation: • ^ • 



. Address: 



City: 



State: . Postal Code: 



Telephone: 



Fax: 



Total Number of Publications Ordered: ^ Total Cost: 

PLEASE MAKE CHECK PAYABLE TO THE FINANCE PROJECT. PREPAID ORDERS ONLY. 

FederalTax ID# 52-184-1608 April 1998 




JL • 4Q 



t 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTICE 

PFPl^omiCTION BA SIS 




This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 



This document is Federally-frmded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 



EFF-089 (9/97) 



021 



10b 




